FM Synth 0.1.5: scale quantizer on MIDI tracks (the author's 2.13 quantizer) - #555

Is this ready to merge?

Thanks for 0.1.5. I tried to bring it up to date with main today, and it can't be merged yet. It needs more than a refresh. Main has moved a lot since 13 October: the Octatrack base now owns the machine table and several MIDI, chromatic and project events, and POLY8 compositions are checked against FM Synth. 0.1.5 hasn't been checked against that yet.

I merged main into your branch locally, moved your notes to src/community/module-changelogs/synth.json, rebuilt the packages in the pinned image and regenerated the catalogue. Your Synth objects come out byte for byte as in your own build. I did not push that to your branch.

What fails on that merge, all without firmware:

• npm run modules:compat: 95 problems. 94 are FM Synth patching stock sites the base now owns (the machine table ot_machines, sequencer rows, MIDI note on/off, chromatic, project load/write), and 1 is a POLY8 composition seam drift at 0x400786c8.
• npm run test: 4 files fail. They are the POLY8 companion tests ("Machine composition seam drift: synth"), module-doctor, synth-release.test.ts, and native-comparisons.test.ts. The last two read the native comparison record, which is still the one for 0.1.2. Re-running it needs npm run module:verify -- synth --os <OS 1.40C file>.
• npm run sdk:check: 20 failures, mostly the module combination tests ("install in every combination through events and tables").

Lint, typecheck, the bundle, the catalogue, licences and the package rebuild pass.

How FM Synth 0.1.5 registers on the base has to be settled on our side before there is anything useful for you to change, so please don't spend time on rebasing it for now. I'll write here again when I know what is needed.

Thanks for the base update! Is the FM Synth 0.1.5 registration blocker now resolved, or should I still wait before rebasing? If it’s ready, what changes are needed before I rebuild and rerun verification—and does the same guidance apply to SY DRUM?

Holding as you asked. When you confirm the new Octatrack base and how a module registers with it, I will re-cut the module from my current 2.18 source instead of 0.1.5, so one update carries the fixes reported in #751, #589 and #557. Do you have an ETA?

Hi Tim, sorry for the wait, and thanks for holding. You can start now. FM Synth 0.1.2 on main shows how a machine registers on the base. Your files stay byte for byte under upstream/, and sdk/machines/octatrack/module-adapters/machine-ports.json maps each of your hooks on a base-owned site to a base event (po_mon to ev_midi_note_on, qz_octkey to ev_octave_key and so on). The rest stay guarded patches. For 2.18: a fresh branch from current main, your files in upstream/, then npm run modules:compat lists every hook that needs a line there. If that part gets fiddly I can do it. The USB package comes from sdk/runtime/loader/build.py --octabam-machine synth. SY DRUM follows the same rule as before: it has to build and work without FM Synth. Two FM Synth PRs are open on the same files: my #895 (0.1.3, card builds only) and Jan's #884 (the #883 fixes). Could you look at #884 and say if you're OK with it or want it in your line? I'll close this one when your 2.18 PR is up.

Thanks, Jannik! I'm starting now: a fresh branch from main with my current line under upstream/, the machine-ports.json lines that modules:compat asks for, and the USB package from build.py. Understood on SY DRUM: I'll make it build and work without FM Synth. I'll answer on #884 separately. Please close this PR when the new one is up.

Hi Jannik, thanks again for unblocking the re-cuts. Before I build SY DRUM, a quick question about which way you'd like it. I'd like people to be able to use SY DRUM and FM Synth in the same project, and I see two ways:

1. SY DRUM inside FM Synth as a second machine, following "ship both features as one module". It would come as an FM Synth update with card builds, like your 0.1.3.
2. SY DRUM as its own module that works without FM Synth and installs beside it. That needs base events for the sites the two share (voice path, LFOs, step locks, scenes and a few keys), and I'd write those PRs for you to review. Its step locks need about 2 MiB, more than a USB package can hold today, so its USB package would also need larger memory regions in the loader.

Both need a fifth slot in ot_machines. Which would you prefer?

A few more things I'd like to check with you before I prepare the PRs:

1. Tuner: its window leaves a close callback and key layer in the host UI, and the base has no removal check for ordinary modules, so a USB removal while it's open could crash. I'd write a base PR that refuses removal while a module's UI is open. Is that the way you'd like it, or do you have another route in mind?
2. FM Synth 0.1.4's step-lock card files hook the engine's job dispatcher at 0x4008485e, where the base has its logger hook. I'd propose an ev_engine_job take event there; the base would then run its logger through that event. OK with you?
3. Two FM Synth 0.1.4 hooks start just before sites ot_machines owns: the LFO destination names at 0x4003bff2 and the chooser commit at 0x40079816. If the base's machine table already does both jobs, I'd let ot_machines' replaces cover them. Does it?
4. Euclid copies 5 bytes from inside FM Synth's step-lock clip hook at 0x4002b654. The same bytes sit unpatched at 0x4002c16b. Is it OK if Euclid copies them from there instead? Its package would then need a rebuild and a version step.

Thanks!

Hi Tim, thanks for asking first.

1. SY DRUM: option 1, inside FM Synth as a second machine. Option 2 means a set of shared base events whose only users would be FM Synth and SY DRUM, plus bigger USB regions, and I'd rather not grow the base for that now. Please keep FM Synth's USB package working; if SY DRUM's step locks can't fit a package yet, tell me what it needs. Both need a fifth row in ot_machines; FRACT needs one too, so I'll raise it in its own base PR and tag you.
2. Tuner: the base already refuses to retire a machine while a reviewed stock UI root still names its image (machine_contributors.c, machine_lifetime.h). I'd extend that check to every module rather than add a second mechanism, and a PR for it is welcome.
3. ev_engine_job: OK. The logger records engine job begin and end at that dispatcher, so a take event it subscribes to fits; please send it as its own base PR.
4. The base owns 0x4003bff4 (the PB table by machine row) and 0x4007981c (the chooser commit, which calls a machine's assign/admit_track). Your hooks start 2 and 6 bytes before those, so they'd overlap. For the commit, use assign/admit_track in your ot_machines row. For the LFO destination names, tell me what that hook does: the base only swaps the PB table pointer.
5. Euclid is mine, so yes, copy the bytes from 0x4002c16b as long as nothing in them is PC-relative; Euclid then needs a rebuild and a version step.

Thanks Jannik, that settles it.

1. SY DRUM goes into FM Synth as a second machine. I'll wait for your ot_machines PR for the fifth row. On USB: FM Synth 0.1.4 is already held at the 0.1.2 package with a declared lag, because the loader refuses the DRAM regions its step-lock and scene tables need (about 2 MiB, 32 KiB, 2.5 KiB and 256 KiB) and its generated includes. SY DRUM adds a 2 MiB lock table and a 64 KiB scene table. I'll write up exactly what the loader would need in the 0.1.5 PR.
2. Tuner: I'll extend the existing check to every module and send it as a PR.
3. ev_engine_job as its own PR, with the base logger subscribing to it.
4. The chooser commit moves to assign/admit_track in my row. The LFO-names hook: on an FM track, LFO SETUP's PMTR formatter picks the PLAYBACK descriptor by the track's machine byte, which is FLEX's in my line, so the hook swaps in FM's own page descriptor and the list reads PTCH RATO INDX FINE FDBK DEC instead of FLEX's names. I'm checking whether your PB-table swap already does that for a contributed row; if it does, the hook goes.
5. Euclid: I'll check the five bytes for anything PC-relative and send the copy change with a rebuild and version step.

One more: my 2.19 quantizer adds seven MIDI-track sites (the MIDI knob, p-lock, key, note, arp trig, preview and held-trig mark) that FM Synth's bundled copy and the standalone quantizer both patch, so the two would overlap. I'm preparing take events for them like the existing ev_knob_store / ev_plock_store. OK with you, or would you rather I handle that another way?

Hi Jannik, three things from building against your answers:

1. The LFO-names hook, your question: on an FM track, LFO SETUP's PMTR formatter picks the PLAYBACK page from the machine table by the track's raw machine byte. A contributed machine's track keeps byte 0 or 1, so your pointer swap at 0x4003bff4 still gives the FLEX page. I tested with and without the hook: PTCH RATO INDX FINE FDBK DEC against PTCH STRT LEN RATE RTRG RTIM. For the base to cover it, the table would have to own the 8 bytes at 0x4003bff2 and resolve the page through mw_page first, as mw_resolve_pb does. Until then the hook stays a declared patch. The chooser commit now goes through assign/admit_track in my row. To keep that card-build hook out of the USB package I added a small dropDetours key to machine_ports.py, which is base-side adapter code in my PR; happy to change the form if you prefer another.

2. For your fifth-row PR: with SY DRUM as a second machine, my module contributes two ot_machines records. machine_registry.c has ROWS 9 and the loader and build.py take one record per module per table, so raising the room alone isn't enough; the base would need to accept two records from one module. Also SY DRUM's SETUP page editor hook at 0x4003a524 (10 bytes) overlaps the table's pointer at 0x4003a526; a setup_edit field or event would cover it. The USB needs you asked for: FM Synth 0.1.4 alone needs four DRAM regions (2 MiB, 32 KiB, 2.5 KiB and 256 KiB) plus generated includes. With SY DRUM it's six regions, about 4.5 MB, and the module image is about 234 KiB against the 128 KiB cap.

3. The removal check extended to every module, as you suggested, works with one catch. After the Tuner closes its window, stock's window manager keeps the close callback in its slot and calls it when the next window takes that slot, so the fail-closed check still refuses removal until the TEMPO window has opened once. Two ways out: (a) at the switch, the base zeroes a closed slot's stale callback that names the leaving module (WCREATE skips a zero callback); (b) the Tuner clears its own slot when it closes. I'd do (b), in the module. OK?

One more: my ev_engine_job draft sits at index 35, which #912 now holds; I'll renumber at merge.

Hi Tim, thanks, that's a lot of useful detail. In the order you asked:

Quantizer, the seven MIDI-track sites. Some may already be covered. #857 added ev_midi_note_key (0x4009fb58, every note from a trig, chord or arp step, before KEY), ev_arp_setup_knob (0x4007a2ec) and ev_arp_setup_text (0x4007a1ec). The MIDI p-lock may also already be ev_plock_store's site (0x40050e60, in the held-trig lock editor). Could you list the seven addresses so we can see which are left?

For the rest, I'd rather not add events whose only users are two copies of your own quantizer. FM Synth doesn't need MIDI-track quantizing for its own tracks. So please put that part in Scale Quantizer only, and keep FM's bundled copy to what FM tracks need (the events it already uses). Sites that only Scale Quantizer patches then stay ordinary guarded patches. We add an event when a second module needs one. One thing to watch: #954 (ARP KEY LOCKS) also takes ev_midi_note_key, so only claim the tracks you actually quantize.

1. LFO SETUP's PMTR names. You're right. That instruction indexes the table by the track's raw byte, and a machine's track keeps 0 or 1, so the pointer swap gives FLEX's page. Analog BD has the same problem, so it's the base's to fix: ot_machines takes the whole instruction at 0x4003bff2 and resolves the page through mw_page, as mw_resolve_pb does. A small base PR from you is welcome, or I'll do it. Until then please don't keep the hook as a patch. It overlaps a site the table owns, so modules:compat refuses it. The package just shows FLEX's names there until the fix lands.

On dropDetours: I'd rather not add a new key. replaced() in build.py only accepts a table detour that starts inside an owned site. Widen it to a detour that overlaps one, and list the hook under the ot_machines contribution's replaces. Then the drop is checked against the table's sites, which a free list isn't.

2. SY DRUM. Thanks for spelling out the one-record limit. I don't want to change the table contract for this. Could SY DRUM be a second engine on FM Synth's own row instead, the way Analog BD opens its engine list (POOL, pool_open, list_draw), with track_page and render picking per track? FM Synth then keeps one record, and the extra rows go to the new machines waiting for one (FRACT, and WAVE in #1028). If there's a reason it has to be its own chooser row, tell me.

For the SETUP editor at 0x4003a524: please check first whether track_page plus the SETUP_ALL flag (on main since #1017) covers it. That's how Analog BD edits its SETUP page over USB. If it still doesn't, I'd take a field in the machine record rather than an event.

USB: thanks for the numbers. The loader won't get multi-MB DRAM regions or a bigger image for now. FM Synth's USB package stays at 0.1.2 with the declared lag, and SY DRUM ships in card builds. One question: a track is one machine at a time, so could FM and SY DRUM share one lock table instead of two 2 MiB ones?

3. Tuner. (b) is good: the Tuner clears its own slot when it closes, and the base's check stays read-only. Please add a line about it to docs/module-guides/standalone.md so the next module with a window knows.

ev_engine_job: yes, take the next free index when it's ready to merge. That's 37 today, but #1023 and #1029 both claim 37 already, so check main at merge time; whoever merges later renumbers and rebuilds the base.

Thanks Jannik, that's all clear, and I'll send the quantizer addresses separately.

On SY DRUM, I'd like to push back a little before changing course. On 26 Oct you wrote:

SY DRUM: option 1, inside FM Synth as a second machine. [...] Both need a fifth row in ot_machines; FRACT needs one too, so I'll raise it in its own base PR and tag you.

I built FM Synth 0.1.5 to exactly that: SY DRUM inside FM Synth, registered as a second machine with its own ot_machines record, waiting only on the extra row. It's built, and the card test image passed every item on my test sheet on my MKI on 26 Oct. Reworking it into an engine on FM Synth's row means redoing the chooser, track_page and render dispatch, plus the setup page and step-lock handling, and then retesting all of it on hardware. That's a lot of rework for something that already works.

You asked whether there's a reason it needs its own row. On the unit, SY DRUM behaves as its own machine: it has its own name in the SRC list and the double-tap list, its own SRC and SETUP pages, and its own step locks and scenes. Users pick it the way they pick FLEX or STATIC. Hiding it behind FM Synth's engine list would make it the one machine you can't choose directly.

Since FRACT and WAVE (#1028) need rows as well, could the base raise ot_machines' room once, to 8 for example? That covers SY DRUM and the machines waiting, without changing how a row works. If that's a problem for the table contract, tell me what limits it and I'll work within that.

On sharing one lock table: I'm open to merging them in a future version. The current build was made and tested before I knew that, with separate tables and their own card files, and card builds have the room, so I'd rather ship it as it is. Since SY DRUM ships in card builds only and FM Synth's USB package stays at 0.1.2, the size doesn't matter for USB yet, so the merge can come in a later update.

You're right, Tim. I said option 1 on 26 Oct and you built to it, so let's keep it: SY DRUM stays inside FM Synth with its own chooser row, and I withdraw the engine-on-FM-Synth's-row suggestion. A machine people pick directly, like FLEX or STATIC, belongs in the list as its own row.

That needs two changes in the base, and I'll make both in one base PR and tag you:

• ot_machines accepts more than one record from one module (the loader, build.py and machine_registry.c);
• the room goes up to eight contributed rows (ROWS 9 → 13), which also covers FRACT (FRACT three-grain track machine (source and native FLEX support) #926) and WAVE (Octatrack: SYNTIZ track machine (SAW, PULSE, ENS, TABLE, MORPH, CHOIR), draft #1028).

Two lock tables are fine for now, since SY DRUM ships in card builds only. Merging them can come in a later update. The SETUP editor question from my last comment still stands: please check whether track_page with SETUP_ALL covers 0x4003a524 before we add a field for it.

Thanks Jannik, that's great, and thanks for the base PR for SY DRUM. Here are the seven quantizer sites, checked against current main (3b79e797): five on MIDI paths and two on audio paths. None of them overlaps a span the base owns, and none is covered by the events you named.

I'll keep those seven as guarded patches in Scale Quantizer only. FM Synth on its own will keep SCALE, ROOT and GLIDE in PROJECT > CONTROL > SEQUENCER, with one SCALE for all FM tracks and no per-track rows, and Scale Quantizer keeps its full per-track rows.

One question before I build that: I'd like FM Synth and Scale Quantizer to install side by side, with one set of scale controls on the page. That needs more SEQUENCER rows than the 8 the base allows today, since Scale Quantizer adds 19, and the two per-track scale masks persisted with the other scale settings. Would you accept a base PR from me that raises the added-row limit to at least 19 and persists those masks? If yes, I'll send it together with both module updates. If not, the two stay declared as conflicting, as they are now.

On the SETUP editor at 0x4003a524: I checked, and track_page with SETUP_ALL doesn't cover it. SY DRUM's handler there also turns edits made while a trig or a scene is held into step or scene locks, and releases a playing step's lock before an ordinary edit; SETUP_ALL only lets encoder E edit the machine's own slot. So the field in the machine record you offered would be the simplest route.

Thanks Tim, that's exactly what I needed.

Scale rows. Let's keep FM Synth and Scale Quantizer declared as conflicting for now, as they are, so no base PR for the extra SEQUENCER rows. FM Synth with one SCALE for all FM tracks, and Scale Quantizer with its per-track rows and the seven guarded patches, sounds right.

SETUP editor. Thanks for checking. I'll add the field to the machine record in the same base PR that lets one module contribute several ot_machines records, and tag you there. The other half is already done: SYNTIZ (#1028) raised the room to eight contributed rows.

This PR still sits on 13 October's main. When you build the quantizer split, please start from a fresh branch off current main, as in my 25 October note. Push it here or open a new PR, whichever is easier for you.

@timhastie, the LFO-names base fix is up as #1080: ot_machines now owns the whole 8-byte lookup at 0x4003bff2 and asks mw_page first, so over USB an FM track's LFO SETUP lists the page your track_page returns. In the emulator an Analog BD track now reads its own names (SYN / TONE / SAT), where it showed FLEX's before. Your card build can keep its LFO-names detour, since card builds have no base: it now starts inside a site the table owns, so list it under the ot_machines contribution's replaces and the USB package drops it.

Thanks Jannik, and thanks for doing the LFO SETUP fix in #1080. I'll list the card build's LFO-names detour under the ot_machines contribution's replaces, as you suggested.

Agreed on keeping FM Synth and Scale Quantizer declared as conflicting for now.

One update on the rows: Scale Quantizer now needs only five SEQUENCER rows instead of 19. They are SCALE, ROOT and GLIDE as before, plus TRACK, which picks a track, and FOLLOW, which sets that track ON or OFF. The per-track settings save through the module's own project lines as before. Five rows fit within the eight contributed rows, so no base change is needed for them.

I'm now working on letting FM Synth and Scale Quantizer install together. They would run one shared quantizer with a single set of those five rows, so there would be no duplicate SCALE, ROOT or GLIDE. I'm building it, and the quantizer split, on fresh branches off current main. Once it passes in the emulator and on my unit, I'll share the results and ask whether the conflict can be lifted for the pair.

Hi Tim, thanks for the update. One thing before you build the install-together version, so it doesn't cost you a rebuild: two modules can't share one quantizer. If FM Synth and Scale Quantizer run a single quantizer with one set of rows when both are installed, one hooks into the other's code and rows, and a module may not hook into another module's code, rows, tables or files. Shared code would have to go into the base or the shared SDK first. So let's keep them declared as conflicting, as we agreed on 30 October, and send the FM Synth re-cut and the Scale Quantizer split as their own PRs off current main. I'll close this one when the FM Synth PR is up.

Generated by Claude Code

Hi Jannik, understood on 31 October: the shared-quantizer build is withdrawn and the two stay declared as conflicting. I'd still like users to be able to have FM Synth and Scale Quantizer on one unit, so here is the dilemma and the routes I can see. I'd like your view on which to take, and whether there is one I haven't thought of.

Why FM Synth keeps a quantizer: its chords are built on scale degrees, its chromatic keys and legato follow the scale, and GLIDE slews between scale notes. Without the scale in FM Synth, mono tracks would still snap through the PTCH word, but chords, legato and GLIDE would lose the scale, and I don't want to take that from people using it.

So both modules need the same scale state and key sites. Card builds have no base, so I've marked what each route gives.

1. One module: FM Synth carries the full quantizer for all tracks, as the one-module case your guide allows. Scale Quantizer stays as the standalone, the two stay conflicting, and card users who want both pick FM Synth. No base work. It goes against your 29 October ask to keep MIDI quantizing in Scale Quantizer only, so only with your OK.

2. A base-owned scale: the base keeps SCALE, ROOT, GLIDE, TRACK and FOLLOW as its rows, the saved bytes and project lines, and exports one accessor, "snap this note for this track", as ev_trig_muted does for muting. FM Synth drops its copy and asks the base; Scale Quantizer keeps the sample and MIDI track snaps. USB only; on a card the pair stays conflicting. I'd write the base PR myself off current main and wait my turn.

3. Both: route 1 on the card, route 2 over USB.

4. Leave it: keep the conflict, send the two PRs as agreed, and point people who want both at my own Octatrick build.

Which would you take? Or if you have another idea let me know. Either way the FM Synth re-cut and the Scale Quantizer split come first, off current main.

Hi Tim, thanks for laying the routes out so clearly. Route 4 for now: keep the two declared as conflicting and send the FM Synth re-cut and the Scale Quantizer split as agreed. Route 1 would put MIDI quantizing back into FM Synth, and I'd rather keep that in Scale Quantizer.

Route 2 is the right shape once more modules want it: a base-owned scale with one "snap this note for this track" accessor could also serve VECTOR and the MIDI generators. Could you open it as a hook proposal issue, with the routes and the modules you see using it? We'll build it when a third module needs it. Card builds keep the conflict either way.

Thanks Jannik. Route 4 it is: the FM Synth re-cut and the Scale Quantizer split come as two PRs off current main, declared conflicting, with the Tuner as a third. The hook proposal for the base-owned scale is #1145, with the routes and the modules I see using it.

The FM Synth 0.1.4 re-cut is up as #1146, off current main, with Scale Quantizer 0.1.4 as #1147 and Tuner 0.1.0 as #1148. This one can close whenever you like.

Thanks Tim. Closing this one now that the re-cut is up as #1146, with Scale Quantizer 0.1.4 in #1147 and the Tuner in #1148. The review carries on there.

Generated by Claude Code